1. Field of the Invention
The present invention relates to an image sensor chip with illuminant estimation functionality, and more particularly, to an image sensor such as that equipped with an additional group of selectively transmissive filters for illuminant estimation, and to an associated illuminant estimation method for an image sensor.
2. Description of the Prior Art
A conventional camera system is typically equipped with a main lens for imaging, an infrared-cutoff (IR-cut) filter (e.g. a separate piece of optics, or a photo resist layer of color filters on photo diodes) to block the invisible light for color fidelity, and a color image sensor to receive the illuminated spectral incident through the main lens and the IR-cut filter. The color image sensor is typically equipped with three sets of color pixels respectively occupying three different spectral bands in the visible for color image acquisition. According to the related art, the Bayer pattern is the most famous and widely used. For example, a group of 2×2 pixels can be its fundamental block, whose layout may correspond to a red-green (R-G) pattern in the first row and a green-blue (G-B) pattern in the next row. The Bayer pixels implemented according to the Bayer pattern mentioned above may have its own spectral responses respectively corresponding to the three sets of color pixels. As the IR-cut filter is utilized, the aforementioned spectral responses respectively corresponding to the three sets of color pixels are converted into the resultant pixel spectral responses. Obviously, the spectral responses possess significant errors from CIE sRGB color matching functions. Color correction by a linear matrix may be employed to try solving this problem, but the approximation error (or the color reproduction error) still inevitably exists.
According to the related art, some conventional methods (see Henker et. al., “Algorithmic Infrared Correction for CMOS Color Sensors”, and Henker et. al., “Concept of Color Correction on Multi-Channel CMOS Sensors”) suggest using a multi-band color filter array (CFA) (which has more than three channels) to improve the approximation. In addition, spatial resolution is addressed by some other conventional methods (see Hirakawa et. al. “Spatio-Spectral Color Filter Array Design for Optimal Image Recovery”, and L. Condat, “A New Color Filter Array with Optimal Sensing Properties”), and new CFAs (with multi-channel arrangements) were proposed to reduce aliasing between luminance and chrominance. Additionally, white (or transparent) pixels and IR pixels are introduced into the pixel array by some other conventional methods, respectively, in order to get stronger sensitivity or receive the reflected IR lights of an assistant IR Light Emitting Diode (LED) in dim light/night vision.
Most of the conventional methods seem to focus on CFA designs for “better images”, such as accurate color, high resolution, low false color and good SNR. Another function/task relying on CFA is illuminant estimation/color temperature (CT) estimation, which is essential to make color correction/white balance/lens shading correction adaptive to various illuminants. Among the conventional estimation schemes, the CT estimation based on the received color image data is popularly used due to its low-cost advantage, where no extra spectral meter or sensor is needed. Its mechanism relies on the so-called grey-world assumption, that the statistical reflectance of natural objects appears grey in visible band (e.g. with equal spectra), so that the ratios or differences among the means of color/Bayer channels can be used to estimate CT or distinguish illuminants. Even though, one inevitable limitation is with the CT estimation scheme, “illuminant Metamerism”. Two illuminants of totally different spectra can result in similar color ratios in Bayer image sensor system (e.g. a tri-chromatic image sensor, such as that of RGB, CMY, etc.) An example can be found in U.S. Pat. No. 8,130,292, where the Incandescent light “2” described therein may give color ratios similar to that of the Fluorescent “6” (and “7”) described therein, though their spectra are known to be greatly dissimilar. Moreover, there should exist difference between ideal color correction and lens shading correction parameters for different illuminants, such as the Incandescent light “2” and the Fluorescent “6” mentioned above, and the method based on the CT estimation may choose the same set of parameters for both illuminants and thus increases errors in the case. U.S. Pat. No. 8,049,789 discloses a typical color correction flow using the conventional CT estimation of the related art, while U.S. Pat. No. 6,947,080 discloses a color space for the conventional CT estimation. Similar ideas are used in U.S. Pat. Nos. 7,030,913, 7,974,487, and 7,746,386 with various modifications. For more information, please refer to U.S. Pat. No. 8,149,294, and US Patent Application No. 2013/0093929.
As described above, illuminant estimation is important for color correction and lens shading correction. However, the conventional methods such as the conventional CT estimation methods typically suffer from Metamerism limitation and may lead the conventional color correction to a dilemma in some cases. When the conventional illuminant estimation fails, the conventional color correction is unable to provide satisfactory performance. Thus, a novel architecture and method is required for enhancing the performance of the camera system.